Web Survey Bibliography
Relevance & Research Question: When asking respondents questions with long lists of response options allowing multiple answers response order effects may occur. In Web surveys, primacy effects are to be expected due to their self-administered nature since the respondents’ attention declines when reading long lists of response options. Instructions asking respondents to process all response options to the same extent have only limited effect. Web surveys offer the opportunity to make use of dynamic tools that have the potential to increase the respondents’ attention to such instructions. Among others, tooltips are an effective method to direct the respondents’ attention to specific elements of a question, in particular to instructions.
Methods & Data: The effect of tooltip instructions was tested in a randomized 2x4 between-subjects experiment embedded in a survey among university applicants (n=6.000). Factor 1 included the order in which a long list of 13 response options was presented (original vs. reversed order). Factor 2 referred to four different ways of presenting the instruction (control condition without instruction, static instruction displayed underneath the question, dynamic tooltip instruction appearing each time a respondent hovered the mouse pointer over a response option, and a combined condition integrating both a static and a dynamic tooltip instruction). For each condition we computed response order effects and compared their magnitude in order to assess the effectiveness of the instruction as compared to the control condition.
Results: Findings indicate that the average response order effect as well as the number of significant effects can be reduced using dynamic tooltip instructions. The reduction is about the same as compared to the effect achieved by a static instruction. When combining the two means of presenting the instruction (static and dynamic) the reduction in response order effects is even more pronounced.
Added Value: The study demonstrates the benefits of using dynamic feedback to respondents in Web surveys. Using tooltip technology works as a continuous stimulus throughout the question-answer process. However, dynamic instructions are not a substitute for static instructions, instead, findings suggest a particular effectiveness of using dynamic instructions jointly with conventional instructions.
GOR Homepage (abstract) / (presentation)
Web survey bibliography (96)
- Examining Factors Impacting Online Survey Response Ratesin Educational Research: Perceptions of Graduate...; 2017; Saleh, A.; Bista, K.
- Using Mixed Methods to Research the Professional Development Needs of English Teacher Educators in PCET...; 2017; Eliahoo, R.
- Why Do Web Surveys Take Longer on Smartphones?; 2016; Couper, M. P.; J. J.Peterson, G. J.
- The impact of visual design and response formats on data quality in a web survey of MOOC students; 2016; Maloshonok, N.; Terentev, E.
- Reminder Effect and Data Usability on Web Questionnaire Survey for University Students; 2016; Oishi, T.; Mori, M.; Takata, E.
- Investigating respondent multitasking in web surveys using paradata; 2016; Sendelbah, A.; Vehovar, V.; Slavec, A.; Petrovcic, A.
- Exploring Factors in Contributing Student Progress in the Open University; 2016; Arifin, M. H.
- Mode System Effects in an Online Panel Study: Comparing a Probability-based Online Panel with two Face...; 2015; Struminskaya, B.; De Leeuw, E. D.; Kaczmirek, L.
- Response Effects of Prenotification, Prepaid Cash, Prepaid Vouchers, and Postpaid Vouchers: An Experimental...; 2015; van Veen, F.; Goeritz, A.; Sattler, S.
- Calendar Instruments in Retrospective Web Surveys; 2015; Glasner, T.; van der Vaart, W.; Dijkstra, W.
- Cognitive Testing of Survey Translations: Does Respondent Language Proficiency Matter?; 2015; Schoua-Glusberg, A.; Park, H.; Meyer, M.; Goerman, P. L.; Sha, M.
- Nannies' Knowledge, Attitude, and Management of Food Allergies of Children: An Online Survey; 2015; Greiwe, J. C., Pazheri, F., Schroer, B.
- Is Vague Valid? The Comparative Predictive Validity of Vague Quantifiers and Numeric Response Options...; 2014; Al Baghal, T.
- Improving Survey Response Rates in Online Panels Effects of Low-Cost Incentives and Cost-Free Text Appeal...; 2014; Pedersen, M. J., Nielsen, C. V.
- The New Majority: How Will Latino Youth Succeed in the Context of Low Educational Expectations and Assumptions...; 2014; Gomez, C. A., Villasenor, E., Mann, E. S., Mandic, C. G., Valladares, E. S., Mercado, V., et al.
- Testing if Healthy Perfectionism Enhances Academic Achievement in Australian Secondary School Students...; 2014; Thorpe, E., Nettelbeck, T.
- Assessing Within-Household Selection Methods in Household Mail Surveys; 2014; Olson, K., Stange, M., Smyth, J. D.
- Web-based Emergency Department Patient Satisfaction Surveys May Introduce Potential for Bias; 2014; Broadwater-Hollifield, C., Fair, J., Podolsky, S., et al.
- Undisclosed Privacy: The Effect of Privacy Rights Design on Response Rates; 2014; Haer, R., Meidert, N.
- Dropout Rates and Response Times of an Occupation Search Tree in a Web Survey; 2014; Tijdens, K. G.
- Students First Choice – the influence of mobile mode on results; 2014; Maxl, E.
- The relationships among nurses' job characteristics and attitudes toward web-based continuing learning...; 2013; Chiu, Y.-L., Tsai, C.-C., Fan Chiang, C.-Y.
- Evaluating Arts-Based Cancer Education Using an Internet Survey among Alaska Community Health Workers...; 2013; Cueva, M., Cueva, K., Dignan, M., Lanier, A., Kuhnley, R.
- Education in the Responsible Conduct of Research in Psychology: Methods and Scope; 2013; DiLorenzo, T. A., Becker-Fiegeles, J., Gibelman, M.
- Intra-individual variation of extreme response style in mixed-mode panel studies; 2013; Aichholzer, J.
- Knowledge of palliative care: An online survey; 2013; Al Qadire, M.
- Identifying the Research Information Needs of the Racing and Breeding Industries in New Zealand: Results...; 2013; Bolwell, C., Gray, Da., Reid, Ja.
- Educational use of smart phone technology: A survey of mobile phone application use by undergraduate...; 2013; Bomhold, C. R.
- Media Commons: the process and value of centralizing services; 2013; Mestre, L. S.
- Dialogistic Presence on Community College Websites in Nine Megastates; 2013; Shadinger, D.
- Adjusting for bias in a mixed-mode CAWI survey on University students ; 2013; Clerici, R., Giraldo, A.
- An imputation approach for analyzing mixed-mode surveys; 2013; Kim, J. K., Park, Se., Kim, S. Y.
- A walking exercise on a tightrope: trying to balance coverage, probabilistic sampling and the costs...; 2013; Poggio, T.
- Validity of Measuring Educational Attainment - Education and Value Orientation in the European Values...; 2013; Ortmanns, V.
- Educational attainment in cross-national surveys: instrument design, data collection, harmonisation...; 2013; Schneider, S.
- Eliminate Item Non-Response: The Effect of Forcing Respondents to Answer in Web Surveys; 2013; Leach, L. M.
- Building a History: Collecting Comprehensive Employment Data in a Web-Based, Multi-Mode Survey; 2013; Cominole, M., Bennett, C., Caves, L.
- Use of Drag-and-Drop Rating Scales in Web Surveys and Its Effect on Survey Reports and Data Quality; 2013; Kunz, T.
- The Impact on Web Survey Drop-Out Rates of Page Number Progress Indicators Used Throughout, Near the...; 2013; Walston, J. T., Cunningham, B., Medway, R.
- Utilizing the Web in a Multi-Mode Survey; 2013; Venkataraman, L.
- Speed (necessarily) doesn't kill: A new way to detect survey satisficing; 2013; Garland, P., Chen, K., Epstein, J., Suh, A.
- Using interactive feedback to enhance response quality in Web surveys. The case of open-ended questions...; 2013; Emde, M., Fuchs, M.
- Reducing Response Order Effects in Check-All-That-Apply Questions by Use of Dynamic Tooltip Instructions...; 2013; Kunz, T., Fuchs, M.
- Ethics, privacy and data security in web-based course evaluation; 2013; Salaschek, M., Meese, C., Thielsch, M.
- Medical students’ exposure to Urology in the undergraduate curriculum, a web based survey; 2012; Derbyshire, L. F., O’Flynn, K. J.
- Social research in online context: methodological reflections on web surveys from a case study; 2012; Pandolfini, V.
- What can be said about quality in the Central Population Register based on a self-completion survey...; 2012; Falnes-Dalheim, E., Pedersen, H. E.
- Boosting Web pick-up Rates by referring to Compliance Principles ; 2012; Falnes-Dalheim, E., Haraldsen, G., Sundvoll, A.
- Ebook readings jumps, print book reading declines; 2012; Rainie, L., Duggan, M.
- Prenotification, Incentives, and Survey Modality: An Experimental Test of Methods to Increase Survey...; 2012; Tepper, J. R., Jacob, B.